Ground Truth in MS Lesion Volumetry - A Phantom Study

A quantitative analysis of small structures such as focal lesions in patients suffering from multiple sclerosis (MS) is an important issue in both diagnosis and therapy monitoring. In order to reach clinical relevance, the reproducibility and especially the accuracy of a proposed method has to be validated. We propose a framework for the generation of realistic digital phantoms of MS lesions of known volumes and their incorporation into an MR dataset of a healthy volunteer. Due to the absence of a “ground truth” for lesions in general and MS lesions in particular, phantom data are a commonly used validation method for quantitative image analysis methods. However, currently available lesion phantoms suffer from the fact that the embedding structures are only simplifications of the real organs. We generated 54 datasets from a multispectral MR scan with incorporated MS lesion phantoms. The lesion phantoms were created using various shapes (3), sizes (6) and orientations (3). Since the common gold standard in clinical lesion volumetry is based on manual volume tracing, an evaluation is carried out from both a manual analysis of three human experts and a semiautomated approach based on regional histogram analysis. Additionally, an intraobserver study is performed. Our results clearly demonstrate the importance of an improved gold standard in lesion volumetry beyond manual tracing and voxel counting.